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STRONG INVARIANCE PRINCIPLES FOR DEPENDENT 
RANDOM VARIABLES 

By Wei Biao Wu 

University of Chicago 

We establish strong invariance principles for sums of stationary 
and ergodic processes with nearly optimal bounds. Applications to 
linear and some nonlinear processes are discussed. Strong laws of 
large numbers and laws of the iterated logarithm are also obtained 
under easily verifiable conditions. 

1. Introduction. Strong laws of large numbers (SLLN), laws of the iter- 
ated logarithm (LIL), central limit theorems (CLT), strong invariance prin- 
ciples (SIP) and other variants of limit theorems have been extensively stud- 
ied. Many deep results have been obtained for independent and identically 
distributed (i.i.d.) random variables. With various weak dependence condi- 
tions, some of the results obtained under the i.i.d. assumption have been 
generalized to dependent random variables. See Ibragimov and Linnik [29], 
Stout [55], Hall and Heyde [25], Lin and Lu [35], Doukhan [16] and the 
collection by Eberlein and Taqqu [18] among others. 

The primary goal of the paper is to establish a SIP for stationary pro- 
cesses. To this end, we shall develop some moment and maximal inequalities. 
As our basic tool, a new version of martingale approximation is provided. 
The martingale method was first applied in Gordin [21] and Gordin and 
Lifsic [22] and it has undergone substantial improvements. For recent con- 
tributions see Merlevede and Peligrad [38], Wu and Woodroofe [67] and 
Peligrad and Utev [40] , where the central limit theory and weak convergence 
problems are considered. The approximation scheme acts as a bridge which 
connects stationary processes and martingales. One can then apply results 
from martingale theory (Chow and Teicher [7] and Hall and Heyde [23]), 
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such as martingale central limit theorems, martingale inequalities, martin- 
gale law of the iterated logarithm, martingale embedding theorems, and so 
on to obtain the desired results for stationary processes. 

To implement the martingale method, one needs to know how well sta- 
tionary processes can be approximated by martingales. In other words, an 
approximation rate should be obtained. In this paper we shall provide simple 
sufficient conditions for the existence oi L'^ {q > 1) martingale approxima- 
tions as well as approximation rates. A random variable Z is said to be in 
L*? {q > 0) if \\Z\\g := [E{\Z\i)]^/'' < oo. It is convenient to adopt the fol- 
lowing formulation. Let {Ci)i£Z be a stationary and ergodic Markov chain 
with values in the state space X; let g:Xh^Mhea measurable function 
for which E[(7(,^o)] = 0; let the filtration J^k = {■ ■ ■ ,S,k-i,(,k), k £Z. Write 
Sn = Sn{g) = J27=i = Qi^i)- This formulation allows stationary causal 

processes. Let be i.i.d. random elements; let ~ (• • • ; £n— ij £n 

) and 

Xn = g{in)- Then {Xn) is a causal process and it naturally falls within our 
framework. As an important category, causal processes have been widely 
used in practice. Asymptotic results on Sn are useful in the related statisti- 
cal inference. The L'^ martingale approximation, roughly speaking, is to find 
a martingale M„ with respect to the filter such that M„ has stationary 
increments (martingale differences) and the approximation error HS'n — Afnilg 
is small. 

The paper is structured as follows. Section 2 presents an explicit con- 
struction of the approximate martingale Mn and a simple and easy-to-use 
bound of the approximation error \\Sn — Mn\\q- Using those basic tools, we 
establish in that section various strong laws of large numbers and invariance 
principles. Some of the results are nearly as sharp as the corresponding ones 
developed under the i.i.d. assumption. Applications to stationary causal pro- 
cesses are given in Section 3. An interesting feature of the limit theorems 
in Section 2 is that, besides the necessary moment condition ^(.^o) £ L'^-, 
they basically rely on the magnitude of On,q = ||IE[5'(Cn)|Co] - ^[^(Cn)!?-!] Ilg, 
n > 0. For stationary causal processes 6n,q is closely related to the physi- 
cal and predictive dependence measures proposed in Wu [64]. It is shown in 
Section 3 that, for causal processes, those conditions can be easily checked. 
Applications to linear processes with dependent innovations are discussed 
in Section 3.2. Proofs are given in Section 4. 

We now introduce some notation. Recall Z G (p > 0) if \\Z\\p = 
[E(|Z|^)]^/P < oo and write \\Z\\ = \\Z\\2- For ease of reading we list the 
notation that will be used throughout the paper: 

• a stationary and ergodic Markov chain. 

• Sn = Snig) = ^27=1 where Xi = g{S,i) and g is a measurable function. 

• J-f; = (. . . , , i^fc) . 

• Projections PfcZ = E(Z|J^fc) -E(Z|J^fc_i), ZeL^. 
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Let Dk = J^i^k'^kdiCi) if the sum converges almost surely. 



• Mfc = Eii A, Rk = Sk- Mfc, A; > 0. 



• 0n,q=\\rog{^n)\\q,n>O. 

• ^n,q = ELo 0i,q- Let 61^,9 = = Am,q if ?7T- < 0. 

• Define the tail @rn,q = Y^iZm ^i^q if ^oo,g = limm_^oo Am,g < oo. 

• Bq = 18g3/2(g _ i)-i/2 if g g (1^ 2) U (2, oo) and Bq = liiq = 2. 

• B: the standard Brownian motion. 

By the Markovian property, for n > 0, Po5(^n) = E[5(Cn)|Co] - ]E[5(C„)|C-i]. 

2. Main results. Basic tools and some useful moment inequalities are 
presented in Section 2.1. Sections 2.2 and 2.3 contain LIL, Marcinkiewicz- 
Zygmund and other forms of laws of large numbers. These results will be 
useful in proving SIP in Section 2.4. 

2.1. Inequalities and martingale approximations. Theorem 1 provides 
moment inequalities and an L'^ martingale approximation for Sn = EiLi g{ii)- 
The theorem is proved in Section 4. 

Theorem 1. Assume that E[(jr(^o)] = and g{^o) G L'^ , q> I. Let q' = 
min(2, q). Then (i) 

oo 

(1) \\Sn\\'i<Bl' Y.{Ai+r.,q-A^,qr'. 



(ii) Assume additionally that 

oo 

(2) eo,g = 5]e»,<j<oo. 

Then Dk ■= Y^^k^kgiii), k £ are stationary and ergodic L'^ martin- 
gale differences with respect to {Tk) and the corresponding martingale = 
J2i=i satisfies 

n 

(3) WS^-MnWilk'iBiY.Ql^. 

(iii) Lei S"* = maxfc<„ ISfcl . Then under (2) , 

(4) \\Sl\\q<^n'I^Q..q. 

q — 1 

Theorem 1 has two interesting features. First, it provides simple moment 
bounds for Sn- Moment bounds for sums of random variables play an im- 
portant role in the study of their convergence properties. Second, it presents 
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an explicit construction of approximating martingales as well as the approx- 
imation rate (3) under the natural condition (2). The latter condition basi- 
cally indicates short-range dependence. Hannan [24] proposed condition (2) 
with q = 2 and proved the invariance principle under mixing. Dedecker and 
Merlevede [11] showed that (2) with q = 2 implies the invariance principle 
without the mixing assumption. McLeish [36] obtained an inequality of type 
(4) for mixingales with q = 2. McLeish's result was improved by Dedecker 
and Merlevede [10]. With the error term (3), we can quantify the goodness 
of the approximation and then apply martingale limit theorems. There is 
a well-developed martingale limit theory and many results established un- 
der the independence assumption have their martingale counterparts. These 
two features are useful in studying the strong convergence of stationary pro- 
cesses. 

We now discuss the issue of the uniqueness of the approximation. Observe 
that (2) and (3) imply HiS^ — Mn\\q = o{n^^'> ). It is easily seen that, if g > 2, 
then such a martingale construction is unique in the sense that if there is a 
martingale = X^iLi with stationary and ergodic martingale differences 
D- with respect to the filter Ti such that \\Sn — = o{^yn), then we 

necessarily have Di = D[ almost surely. To this end, note that ||Mj!j — M„||q < 
\\M'^-Sn\\q + \\Sn-Mn\\q = o{^). Then || - M„ || = o( V^) since g> 2. 
Consequently \\Di — D'l\\^/n = o{-\/n) and ||-Di — || = 0. 

Remark 1. In the case of q = 2, the construction of the approximat- 
ing martingale {Mk)k>o also appeared in Hall and Heyde ([23], page 132), 
Woodroofe [62] and Volny [60]. To the best of our knowledge, the rate of 
approximation (3) with a general g > 1 is new. Inequality (3) with g > 2 is 
needed in our derivation of the laws of the iterated logarithm and strong 
invariance principles; see Theorems 2, 3 and 4. 

Remark 2. Yokoyama [71] considered a related problem. The basic as- 
sumption imposed in his paper is that there exists a martingale M„, with 
stationary and ergodic increments such that 

- Mn\U = 0{(nlog2n)^/2[(log?i) • • • ilog„,n)'+']-'} 

for some 6 > and m>l. Here log2 n = log log n and log^ n = log(log„j_^ n). 
Under such an assumption, Yokoyama obtained a LIL for Sn- For linear pro- 
cesses with i.i.d. innovations, due to the special linearity structure, approx- 
imating martingales can be easily constructed; see Section 3 in his paper. 
However, the latter paper did not address the issue of how to construct 
approximating martingales for general stationary processes. For nonlinear 
processes, it is not straightforward to construct such approximating mar- 
tingales with the desired rate. The explicit construction in Theorem l(ii) 
overcomes such restrictions. Some examples are presented in Section 3. 



STRONG INVARIANCE PRINCIPLES 



5 



Remark 3. In the special case q = 2, Wu and Woodroofe [68] studied 
the existence of triangular stationary martingale approximations. Such a 
martingale approximation scheme means that, for each n, there exist sta- 
tionary martingale differences Dni,Dn2, ■ ■ ■ such that 



(5) 



max \\Sk 

l<k<n 



nk I 



where Mnk = J2i=i ^m- Wu and Woodroofe [68] showed that (5) is equiva- 
lent to ||E(5„|^o)|| = cdlS'nII). Under the latter condition, \\Sn\\ has the form 
yjnh{n), where /i is a slowly varying function. We say that a function i is 
slowly varying if for any A > 0, Iima;_>oo^(Ax)/£(x) = 1 (Feller [20], page 275). 
Their L? martingale approximation leads to a necessary and sufficient con- 
dition for conditional central limit theorems. The martingale approximation 
(3) differs from (5) in that it allows a general q> 1 and that the single-array 
sequence {Mk)k>Q itself is a martingale with stationary increments. 



Remark 4. Inequality (1) is applicable to cases in which O^^q decays 
slowly. For example, if 9n,q = (l-|-n)~^, 1/ q' < [3 then (1) gives 1 1 S-fi 1 1 q — 
0{n^li'+^-l^).ln this case (ii) and (iii) are not applicable since (2) is violated. 
Corollary 3 contains an application of (1) to strongly dependent processes. 

Inequalities concerning maxfc<„ [Sfcl are important devices in establishing 
limit theorems for Sn- Proposition 1 provides simple maximal inequalities 
which are useful in the study of strong convergence of stationary processes. 
Corollary 1 presents an almost sure version of the martingale approximation. 
Similar maximal inequalities appeared in the literature; see Menchoff [37], 
Doob [15], Billingsley [5], Serfling [50], Moricz [39] and Lai and Stout [33]. 



Proposition 1. (i) Let q> I and Zi, 1 < i < 2^, be random vari- 
ables in , where d is a positive integer; let Sn = Zi + • • ■ + Zn and S* = 
maxj<„|S'j|. Then 



(6) 



\'J2d \\q 



d 

<E 

r=0 



'S'2'-(m-l)llg 



.m=l 



1/? 



(ii) Let {{Yk^^)kez,4' ^ ^ class of centered stationary processes in 

L'^,q > 1; namely for each </> G (^fc,</))feGZ stationary and Yq,;/) £ L'^! ls.t 
Sn,s = Yi^s^ 'tYn^s. Then 



(7) 



E* 



max sup |5'fc^^|'^ 



1/9 



d . 
<^2('^-J')/'?Je* 
i=o 



SUp|S'2J,<^| 



i/a 



where E* is the outer expectation: WZ = \n.i{E,X:X > Z,X is a random 
variable}. 
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(iii) Let {Zi)i^z be a stationary process and Zq £ L'' , q> 0. Then for all 
S>0, 

oo oo 

(8) ^(-52^ ^ 2^'^"'^) ^ 2r''A^+S where \ = ^(2-^'||52. 11^)^/^''+^^ 
fe=o i=o 

Remark 5. MenchofF [37] and Doob [15] considered the special case of 
(6) with q = 2 and uncorrelated random variables. A maximal inequality 
for stationary processes with g = 2 is given in Wu and Woodroofe [68] . 
The current form (6) allows a general q > 1 and nonstationary processes. 
Inequality (7) is a useful variant of (6) and it is applied in Wu [65]. 

Corollary 1. Let g{^i) e L'^, q> 1, E[5r(^i)] = and assume (2). Let 
Rk = Sk- Mk, where Mk = ELi A and Dk = TZk^kgUi)- Then i?„ = 
Oa.sX''^^^'^) under 

oo 

(9) A;— i°[i'(''+4)/(29+2)] 99/(9+1) ^ 

k=l 

Proof. If 1 < g < 2, since Qn,q is nonincreasing in n, 

oo / 2J \ l/(g+l) oo / j \ 1/(9+1) 

j=0\ 1=1 I j=0 \k=0 I 

oo i 

< Y 2"^'/(9+l) J2 2''/(9+l) 09/(9+1) 
j=0 k=Q 



oo oo 

< 



Y E 2"^'/(''+i)2'^/(9+i)e'*'/('*'+^) 

k=Oj=k ''^ 
oo oo 

k=0 n=l 

By Theorem l(ii), E^o(2"^H^2^ llp^''^''"^^^ < oo. By Proposition l(iii) and 
the Borel-Cantelli lemma, we have Rn = Oa.s.(f^'^^'^)- The other case q > 2 
similarly follows. □ 

Remark 6. If e„,q = 0[n'''"'^°'^/i-^/^'>{logny], r < -1 - then (9) 
holds. 

Remark 7. Let q > 1. As in Gordin and Lifsic [22], we have ||-Rn||q = 
0(1) if 

oo 

(10) Y^igi^o] ^ H{Co) (say) in 

i=0 
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by letting Di = H{^i)- E[H{^i)\J^i^i]. By Proposition l(iii) and the Borel- 
Cantelli lemma, Rn = o^.s.i''^^^'')- The two sufficient conditions (9) and (10) 
have different ranges of applicability. Let g{Cn) = Y^iZii'^^n-i, where e„ G 
L*^ are i.i.d. random variables with E(e„) = and E(|e„|^) = oo for any r > q. 
If a > 1, then en,q = ©(n""), G^,, = 0(ni-°) and (9) holds, while (10) re- 
quires a > 1 + 1/q. On the other hand, however, let g{(,n) = J2^i{~^y^~^n-i, 
l/q< P < 1; then (9) is violated while (10) holds. 

2.2. Laws of large numbers. The ergodic theorem is probably the best 
known result on strong convergence of stationary processes. Let (X^) be a 
stationary and ergodic process with E(|Xo|) < oo and X„ = ^27=1 ^i/f^'i then 
Xn — E(Xo) almost surely. However, the convergence of X„ — E(Xo) — > 
can be arbitrarily slow even \iXk is bounded (Krengel [32]). Here we consider 
the rate of Xn — E(Xo) under conditions on the decay rates of 9n,q- 
Corollaries 2 and 3 are easy consequences of Theorem 1 and Proposition 1 . 

Corollary 2. Let g{ii) G L'^, q> I, E[5i(^o)] = and i be a positive, 
nondecreasing slowly varying function. 

(i) Assume Qn,q = ©[(logn)"""], < a < 1/q, q>2 and 

Then 5„ = Oa..s.[\/n£{n)]. 

(ii) Assume (2) with l<q<2 and EfcLi^"H2^) < oo. Then Sn = 
Oa.s.[ni/^^(n)]. 

(iii) Let 1 <q <2 and assume (9). Then Sn = o^,s.{n^^'^) ■ 

Remark 8. Let 5 > 0. Then (11) holds if l{n) = (logn)V9-" x 
(loglogn)(^+^)/'i'. The function tq{n) = (log?i)^/«(loglogn)(^+'')/« satisfies (n). 
By (n), if 60,2 < 00, then Sn = Oa.s.[\/ra£2(n)]. 

Remark 9. Let 1 < q<2. By the martingale Marcinkiewicz-Zygmund 
SLLN (cf. Corollary 3 in Tien and Huang [58] or Woyczyhski [63]), (10) im- 
plies Sn = Oa.s. (?^^^''); see Remark 7. Dedecker and Merlevede [12] considered 
Banach- valued random variables. 

Proof of Corollary 2. (i) Let Mk = EiLi A and Rk = Sk- Mk. By 
(3) of Theorem 1, ||i?„||q = 0[n-'^'^(logn)~"]. By (11) and Proposition 1, 



1 



E 2fc'?/2£9(2fc)^[™'^''1 - 2^9/2^9(2'=) 
(12) 



1 



.3=0 
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which by the Borel— Cantehi lemma imphes maXj<;2fc 

and consequently |i?n| = Oa_,s,.[\/nl{n)\ since d. is slowly varying. By (11) and 
since ^ is nondecreasing, 

^ \P(Ok\]q - 
k 



; [^(2fc)]9 - [i{2^)]i 1 - aq [i{2^)]<i ' 



So (logn) /"^ " = 0[i{n)] and (i) follows from Stout's [54] martingale LIL 

(13) limsupifc^ f^" = \\Di\\. 

n-»oo V 2n log log n 

(ii) By Theorem l(iii), = 0(n). So E^i 2~''''/^i~i{2'')\\S;, ||p < oo. 
By the Borel-Cantelli lemma, since i is slowly varying, 5^ = Og,,s.[n^^'^^in')]. 

(iii) By Corollary 1 and the martingale Marcinkiewicz-Zygmund SLLN, 

Sn = Rn + Mn = 0^.sXn^/'i). □ 

Remark 10. Let q = 2 and £*{n) = Yli=i[i^{i)]~^ , where 1^ is a positive, 
slowly varying, nondecreasing function. Zhao and Woodroofe [72] proved 
that Rn = Oa.s. [\/n£*{n)] under 

oo . 

(14) ^r3/y^«(logi)||E(S,|^o)|| < oo. 



i=l 



Hence Sn = Os,,s.W n£* (n)] if log log n = o[i*{n)]. Their result does not cover 
our Corollary 2(ii). On the other hand, however, Corollary 2(ii) does not 
allow functions like i{n) = y/logn. 

We now give an example where (14) is violated while Go,2 < oo. Let 
Ei be i.i.d. with E(ej) = and E(e?) = 1 and = J2i^oO'i^n-i, where 
{O'i)i>o are real coefficients. Let J^i = {. . . ,ei-i,ei). Then ||E(S'j|.Fo)|p = 
X;°^o(aj+i + • • • + flj+j)^. Let Qi = A;-3/2 if i = 2'=, /c G N, and = if oth- 
erwise. Since Oj are nonnegative, ||E(5i|.7xi)|| is nondecreasing. Note that 
||E(S2fe|^o)f >2'=Ez=A:4 >2'=A;-V2. So E.=i ^"'/'l|E(5,|.7^o)|| = oo since 
E^i 2-'=/2 (2^=^-2) V2 = oo. Clearly 60,2 < 00. 

Corollary 3. Let q> I and q' = min(2,g). 

(i) Assume \\E[g{^n)\^o]\\q = 0(?^"''), < < 1. Lei 7 = max(l-7?, 1/g')- 

(11) // 1 - ry / 1/q', then \\S*Jq = O(n^) and 5„ = Oa.s.(n^(logn)i/5 x 
(log logn)^/*?). 

(12) If I - 7] = 1/q', then \\S*\\q = 0(?i'^logn) and 5.„ = Oa.sX^^"^ x 
(logn)i+V'?(loglogn)2/'?). 

(ii) Let 9n,q = 0[n~^£{n)], where 1/q' < /3 < 1 and £ is a slowly varying 
function. Then \\S*\\q = 0[n'^£{n)] and Sn = Oa.s.[n''^£{n)logn], t = 
l/q' + l-P. 
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Proof, (i) We shall apply the following inequality: there exists a con- 



stant Cq> such that 



j=0 

Peligrad, Utev and Wu [41] proved the above inequality when q >2 [see 
inequality (10) therein], while Wu and Zhao [70] proved it under 1 < q <2. 

Note that ||E(52i|^o)||, < Ef=i Oir^) = 0{j + 2^-^'^). 

(11) Let 1 — r/ / 1/g'. Elementary calculations show that ||S'*d ||g = 0{2'^'^). 
Hence || 5**115 = 0(71^). Using the argument in the proof of Corollary 2(ii), 
the almost sure bound of Sn follows. 

(12) It can be similarly dealt with. 

(ii) By Karamata's theorem and Theorem l(i), A„^q = 0[n^~^£{n)], and 



i=—n i=l+n 
00 

iO(At,,)+ E 0[mi-^£~(i)]'?'=ni+(i-^)''' [£>)]-?'. 



i=l-\-n 

By Proposition l(i), \\S*a\\q = 0[2'^''e{2'^)]. By the Borel-Cantelh lemma, 
(ii) follows. □ 

Remark 11. If (11) holds with a>l/q, then there exists a law of the 
iterated logarithm, a more precise form of strong convergence. See Theorem 
2(i). 

Remark 12. Let q > I and q' = min(2,g). Roughly speaking, since 
©0,5 < 00, Corollary 2 deals with short-range dependent processes. The or- 
der of magnitude of Sn is roughly n^/'^', up to a multiplicative slowly vary- 
ing function. Corollary 3 allows long-range dependent processes. For ex- 
ample, if g = 2 and Xn = J2iZo(^i^n-i, where En are i.i.d. with E(e„) = 0, 
E{el) = 1 and a„ = n"^, 1/2 < (3 < 1, then ||E(X„|J^^o)ll = 0{n^/^-'^) and 
Sn = Oi,.s.{n^/^~^ logn). 

Remark 13. With the help of the maximal inequality (6), Corollary 2 
can be generalized to nonstationary processes. Let (Xj)jgp^ satisfy lE(Xj) = 
and EdXil") < 00 for ah i E N; let 5„ = Xi + • • • + X„ for n G N, So = 
and S* — maXj<72 1 I . Assume that there exists a nondecreasing func- 
tion / such that /(O) = 0, liminf„_>oo /(?T')//(2n) > and — Sm]\q < 
/(n) - f{m) for all integers n > m > 0. By (6), WS^^Wg < d[/(2^)]V9. If 
there exists a positive, nondecreasing slowly varying function £{■) such that 
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X]fcLi[^/^(2'^)]'^ < OO, then the argument in the proof of (ii) of Corohary 2 
yields 5„ = Oa.s.[f^^'^ in)i{n)]. In the special case q = 2 and /(n) = n"", cr > 0, 
let i{n) = (log n)^/^ (log log n); then Sn = Oa.s. [?^'^^^^(ra)] . The latter result is 
slightly better than the Gaal-Koksma SLLN (Philipp and Stout [44], page 
134), which states that 5„ = Oa.s.[?W^(logn)3/2+5] fQj. gj^^j^ ^ > q. 

2.3. Laws of iterated logarithm. 

Theorem 2. (i) Assume that g{£^i) e L'? for some q> 2, E[g{^o)] = 
and 

^L(log«)i/^J 

Then for a := \\ J2'^o'Pog{S,i)\\ < oo, we have, for either choice of sign, that 

(16) limsupifc^ l^'" =a 

OO ^/2n log log n 

almost surely. 

(ii) Let g{^i) £ and E[g{^i)] = 0. Then (16) holds under (9) with q = 2. 

LIL for stationary processes has been studied by Philipp [42], Reznik [47], 
Hah and Heyde [23], Stout [55], Rio [48], Volny and Samek [61] and Zhao and 
Woodroofe [72] among others. An interesting feature of Theorem 2 is that 
the conditions (15) and (9) only involve 6n,q and they are easily verifiable 
for causal processes; see Proposition 3(ii). 

Theorem 2 is proved in Section 4. The key idea is to show that Rn = 
Oa.s. ( log log n) . Using the martingale version of Strassen's functional LIL 
(cf. Heyde and Scott [27] or Basu [3]), we can easily obtain the functional 
LIL: let rjn{t), t G [0,1], be a function obtained by linearly interpolating 
Si/\/2n log logn at i = i/n, i = 0,1, . . . ,n; let i;^ be a continuous map from 
C[0, 1] to M. Then under conditions in Theorem 2, 4>{rin) is relatively compact 
and the set of its limit points coincides with (^{F), where F is the set of 
absolutely continuous functions / satisfying /(O) = and /o^[/'(t)]^ < 1- 

Both (i) and (ii) of Theorem 2 concern LIL. They have different ranges 
of applicability. The former imposes q > 2 while in the latter q = 2. On 
the other hand, in some cases, (15) is weaker than (9) with q = 2. Let 
g{^n) = Si^o '^i^n-ii where are i.i.d. {q > 2) random variables and = 
n~i(log?7,)~", n>2, where a> 1. Then @n,2 = 0[{logn)^~'^] and Qn,q = 
0[(log?i)i~"]. It is easily seen that (9) needs a > 5/2, while (15) only needs 
a > 1 + l/q. 

Remark 14. Zhao and Woodroofe [72] obtained a LIL by letting l{i) = 
logi in (14). Their result refines earlier ones by Heyde and Scott [27] and 
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by Heyde [26] in the adapted case. Their LIL and Theorem 2(ii) have dif- 
ferent ranges of appUcabihty. Consider the Unear process example in Re- 
mark 10. Now we let aj = if i = 2^, /c G N, and aj = if otherwise. 
So (9) holds with q = 2 since 62^,2 = 0{k'^). Note that ||E(S'2fe|-Fo)f > 
2*^ E£fc all > 2^k-^/b. Since Y.T=i 2~''l'^k^l'^{2^k~^Y''^ = 00, (14) is violated 
[recall £{i) = logi]. On the other hand, let = z~^(logi)~^/^(loglogi)", 
i > 10. Elementary calculations show that Zhao and Woodroofe's LIL re- 
quires Q < — 1 while our Theorem 2(ii) requires a < —3/2. 

Remark 15. A simple sufficient condition of (15) is 0^^^ = 0(log~^ n), 
which is weaker than the one in Yokoyama [71] even in the special case of 
linear processes. Let g{Cn) =J2iZoai£n-i, where En are i.i.d. and a„ are real 
coefficients. For = 0[n~^(logn)~"] and eo S L'^, 9 > 2, Yokoyama's result 
requires q > 1 + 2/q whereas a > 1 + l/g is enough to ensure (15). 

2.4. Strong invariance principles. The strong invariance principle stud- 
ied here means the almost sure approximation of partial sums of random 
variables by Brownian motions. With such approximations, asymptotic prop- 
erties of partial sums can be obtained from those of Brownian motions. 
Strong invariance principles are quite useful in statistical inference of time se- 
ries and have received considerable attention in probability theory. Strassen 
[56, 57] initiated the study for i.i.d. random variables and stationary and er- 
godic martingale differences. Komlos et al. [30, 31] considered approximating 
sums of i.i.d. random variables by Brownian motions and obtained opti- 
mal results. Motivated by the usefulness of such approximation scheme, re- 
searchers have established many results concerning mixingales and strongly 
mixing processes of various types; see Berkes and Philipp [4], Bradley [6], 
Eberlein [17], Shao [51], Rio [48], Lin and Lu [35], Dedecker and Prieur [13] 
and Philipp and Stout [44] among others. Philipp [43] gave an excellent 
review. 

Here we shall apply the moment and maximal inequalities and SLLN 
developed in Sections 2.1 and 2.2 to study strong approximations of partial 
sums of stationary processes. To state such results, one often needs to enlarge 
the underlying probability space and redefine the stationary process without 
changing its distribution. For brevity we simply say that there is a richer 
probability space and a standard Brownian motion IB such that the partial 
sum process can be approximated by IB with certain rates. 

It seems that our method is quite effective and it leads to nearly optimal 
approximation rates. Let Xq(") = n-^/''(logn)^/^ if 2 < g < 4 and x<j('^) = 
ni/'*(logn)^/2(loglogn)^/^ if g > 4; let Lq{n) = jzi/g (log n) 1/2+1/9 (log log n)^/'?. 
Recall Theorem 1 for Dk = EZk'Pkgi^i)- 
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Theorems. Letg{io)eLi, g > 2, £[5(^0)] = anda=\\Dk\\. Let q* = 
min(g, 4). 

(i) Assume Qn,q* = 0[n^/'^*~^^'^{logn)~^] and 

00 

(17) Y.\\^(^1\^o)-<tX,/2<oo. 

k=l 

Then on a richer probability space, there exists a standard Brownian motion 
B such that 

(18) \Sn-M{a\)\ = 0.,.s.[xq(n)]. 

(ii) Assume Qn,q* = 0{n^/'^''~^/'^) and 

00 

(19) 5]||Po(I?i)|lr/2<oo. 

k=l 

Then on a richer probability space, there exists a standard Brownian motion 
M such that 

(20) |5„-B(cT2n)|=Oa.s.[vH]- 

Remark 16. In Theorem 3, conditions (17) and (19) involve the decays 
of ||E(L)^|jro) — (T^||g*/2 and ||^o(-Df.)||q./2- Interestingly, it turns out that 
for stationary causal processes both quantities have simple and easy-to-use 
bounds; see Proposition 3 in Section 3. 

Theorem 4. Let g{^o) e L'^ , 2 < q < 4, and E[g{^o)] = 0. Assume Qn,q = 
0{n~'^) and ||E(D^|^o) — '^^11^/2 = 0{n~^), 77 > 0. Then on a richer prob- 
ability space, there exists a standard Brownian motion B such that \Sn — 
B((7^?i)| = Oa.s.[?^^^^(logn)^/^], where 7 = max(l — ri,2/q). 

Theorems 3 and 4 give explicit approximation rates. It is interesting to 
note that, when 2 < < 4, the rates (18) and (20) are optimal up to multi- 
plicative logarithmic factors. Komlos et al. [30, 31] showed that, if {Xk)kez 
are i.i.d. random variables with mean and gth moment, q > 2, then the 
approximation rate is o(n^/'^) and it cannot be improved to o{n^) for any 6 < 
1/q. Strong invariance principles with the rates n^/^~^ or o[(n log log n)^/^] 
are widely considered in the literature; see Philipp and Stout [44], Philipp 
[43] and Eberlein [17]. 

SIP plays an important role in statistical inference. For change-point and 
trend analysis see Csorgo and Horvath [8]. Wu and Zhao [69] considered 
statistical inference of trends in time series and constructed simultaneous 
confidence bands for mean trends with asymptotically correct nominal cov- 
erage probabilities. Shao and Wu [52] studied asymptotic properties of the 
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local Whittle estimator of the Hurst (long-memory) parameter for fraction- 
ally integrated nonlinear time series models. In the latter two papers we 
applied the SIP of the form (20) and the rate Oa.s. Vq* {n)] is needed to con- 
trol certain error terms. 

It is unclear whether a bound of the form 0[n^^'^l{n)\, where £ is a slowly 
varying function, can be obtained when q> A. The arguments in Komlos et 
al. [30, 31], which heavily depend on the independence assumption, cannot 
be directly applied to stationary processes. In our proof of Theorem 3 we 
apply Strassen's [57] martingale embedding method. The best bound that 
Strassen's method can result in is X4('t-)- I^i the special case of linear pro- 
cesses, since the approximate martingales are sums of i.i.d. random variables, 
sharp bounds can be derived. 

Proposition 2. Let X„ = YA^QaiSn-i, where En e L'^, q> 2, are i.i.d. 
with mean and the real coefficients an are square summable. Assume that 
Aj := J2i^j ' J ^ 0, exists and 

n oo 

'£a^+ y: iA-A.n)'. 

1=1 i=n+l 

Then on a richer probability space, there exists a standard Brownian motion 
IB such that 

(22) IS'n -B(o-2n)| =Oa.s.(^^^'') where a = \AQ\\\eQ\\. 
In particular, (21) holds under (9), or X^i^i^f < oo, or 

oo oo 

(23) E^E«'<°°- 

i=l ^ j=i 

Proof. Let Mk = Aq Y!1=i efc • Then = M„ - S„ = E^i {^i - Ai^n x 
li>n)£n+i-i- Rosenthal's inequality, ||i?n||<j < C||i2„|| for some constant 
C <oo. Since \\Rn\\ =S„||eo||, by Corollary 1, (21) implies i?„ = Oa.s.(?^^^'^) 
and hence (22) by the Hungarian construction of Komlos et al. Clearly 
< oo implies H„ = 0(1). Under (23), E^=oIE(^n|-^o) converges in 
L^, we have as in Remark 7 that ||i?n|| = 0(1) and then (21). □ 

Phillips [45] studied the consistency problem of log-periodogram regres- 
sion. A key tool in the latter paper is a SIP for linear processes with the 
rate Oa.s.(nO, CG (2/g,l/2) under 

oo 

(24) E^l"«l < °° 

i=l 



(21) E(2" 



< oo where H„ : 
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and Ei G L'^, q > 4. We now compare Phillips' result with our SIP (22). First, 
our result requires q > 2 while q > 4 is needed in Phillips' result. Second, 
Phillips' condition (24) implies Qn,q = 0{n~^) and hence (9). Our conditions 
are much weaker. Third, our bound Oa.s.(f^^^''') is optimal and is sharper than 
Phillips' Oa.s.(?^^)- Akonom [1] obtained a bound op(n^/'^) under (24). 

3. Applications. Let (en)nGZ be i.i.d. random variables and let X„ = 
g{S,n), where ^„ = (. . . , e„_i, e„) and g is a measurable function such that 
Xn is a proper random variable. We can view the random variables £i,i <n, 
as the input to a system, g as a filter and Xn = g{^n) as the output of the 
system. The class of causal processes is huge and it includes a variety of 
nonlinear time series models (Priestley [46], Tong [59] and Stine [53]). 

In this section we shall apply the results in Section 2 to stationary causal 
processes. Due to the structure of the filter J^n can be naturally de- 
fined as the sigma algebra generated by Write ¥.{Z\J^n) =IE(Z|^„). To 
apply the moment inequalities and limit theorems presented in Section 2, 
one needs to verify the conditions therein [cf. (2), (9), (15), (17) and (19)] 
which are based on the quantities \\Vog{^n)\\q, \\E[g{£,n)\J^o]\\q, ll^o(-C'.^)||g 
and ))E(Z)^].7^o) ~ '^^llg- It turns out that for causal processes such quan- 
tities can be effectively handled by the coupling method. Let (e^)nez be 
an i.i.d. copy of (en)nGZ and let = (. . . , e^_]^, e^). For /c > let = 
{^o,ei,...,£k-i,ek) and 4 = (C-i, Eq, ei, • • • ,f fc-i, efe)- In^^ the whole "past" 
^0 is replaced by the i.i.d. copy ^'q, while only couples a single innova- 
tion £q. Let hmiCk) =^[g{^k+m)\Ck] be the m-step conditional expectation, 
m > 1, and write /i(^fc) = /ii(Cfe)- For /c > define 

ak = WHCk) - h{ik)\\q, al = WhiCk) - hm\q, 

(25) 

Pk = uck) - gm\q, = \mk) - gia)h- 

The input/output viewpoint provides another way to look at the depen- 
dence. Wu [64] introduced the physical dependence measure (3k, which mea- 
sures how much the process will deviate, measured by the L'^ distance, 
from the original orbit {g{£.k))k>o if we change the current input Eq to 
an i.i.d. copy e'q. By (27), ll'Po*7(?n)ll5 has the same order of magnitude 
as WKi^o) - K{io)\\q- Note that hn{Co) = '^[gi^nMo] is the n-step ahead 
predicted mean. So w„ = ll/in(^o) — ^n(^o)||g measures the contribution of 
eo in predicting future values. In Wu [64] ujn is called predictive dependence 
measure. The short-range dependence condition Go,q < oo means that the 
cumulative contribution of eo in predicting future values is finite. Dedecker 
and Doukhan [9] proposed E.\g{^k) — g{- ■ ■ ,0,£i, ■ ■ ■ ,£k)\, a similar version of 
See Wu [64] for more discussions on input /output dependence measures. 
Proposition 3 provides bounds for \\VogiCn)\\q, \mg{U)\^o]\\q, ll^o(^^,)||g 
and ||E(Z)^|J^o) — cr^llg based on the quantities in (25). Despite the fact that 
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the martingale differences Z)„ are constructed from g{£,i) in a complicated 

I?- 



manner, there exist simple bounds for ||'Po(-Cn)||g and ||E(L',^|^o) — c^' 



Proposition 3. Let 5(^0) ^L'^, q> 1, q' = min(2,g), and £[5(^0)] = 0. 

(i) Let k>0. Then au < 2(3k+i and al < 2(31^-^. 

(ii) For 71 > 1 we have 

\\\hn{io)-K{eo)\U<\n9{in)M\U 

(26) 

<mm[||/i„(^o) - ^n(Co)llg'«n-i] 

and 

(27) i||/i„(eo) - /in(e~o)||g < \\Vo9{in)\U < min[||/i„(eo) - /l„(^~o)||g, "n-l]. 

(iii) Let q>2 and EZi^i < 00. Then Dt = Y.Zk'Pkgi^i) G L'' . Addi- 
tionally, for A; G N, 

00 

(28) mDl\io) - AUn < 8c,/?fc* + 8cgJ2mm{a*,a,_k), 

i=k 

wherea=\\Dk\\ and Cq = \\Dk\\q, and 

00 

(29) \\Vo{Dl)%/2 < 8c + 8c, ^ a,. 

i=k 

Remark 17. In Proposition 3 (iii), the results are expressed in terms of 
the one-step conditional expectation h{^k) =^[giS.k+i)\(,k]- It is straightfor- 
ward to generalize them to the m-step conditional expectations hm{Ck) = 

n9i(k+.^Mk]. 

Corollary 4. Let 5(^0) ^ L'^, 2<q<A, and E[5'(^o)] = 0. Assume 

00 

(30) ^(/3fc + A;afc)<oo. 

fe=i 

Then (20) holds. In particular, (30) holds if 

00 

(31) < CX). 

k=l 

Corollary 4 is an easy consequence of (ii) of Theorem 3 and Proposition 
3 in view of @n,q < J2k^=n^k-i = 0{n~^), by (27) and (30). By Proposition 
3(i), ctfc < 2/3fc+i, so (31) imphes (30). Corohary 4 provides simple sufficient 
conditions for strong invariance principles. The quantities and Pk are 
directly related to the data-generating mechanisms. 
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3.1. Transforms of linear processes. Let X„ = J2i^o ^j^n-j> where ej. are 
i.i.d., Ek G L'^, 9 > 1, and (an)n>o is a real sequence. Let ii' be a Lips- 
chitz continuous function; namely, there exists a constant C < oo such that 
\K{x) - K{y)\ < C\x - y\ for all x,yeR. Let g{Cn) = K{Xn) - E[i^(X„)]. 
Then en,q <Pn = 0{\\an{eo - e'o%) = 0(|a„|). So (31) holds if EZi ^Kl < 
oo. 

3.2. Linear processes with dependent innovations. Let {£k)kez be i.i.d. 
random elements and let (a„)„>i be a sequence of real numbers; let 

oo 

(32) Xn = ^air]n-i where = G(. .., e^-i, en) 

and G is a measurable function. Linear processes with dependent innovations 
have attracted considerable attention recently. Models of such type have 
been proposed in econometric time series analysis. See Baillie, Chung and 
Tieslau [2], Romano and Thombs [49], Hauser and Kunst [25], Lien and 
Tse [34] and Wu and Min [66] among others. Appropriate conditions on 
the innovations are needed for the asymptotic analysis of (X„). Let q > 2. 
Assume that there exist C > and r G (0, 1) such that for all n G N, 

(33) E[mn)-GiCW]<Cr\ 

where ^* = (^q, ei, . . . , e„). The property (33) is called the geometric- moment 
contraction condition (Hsing and Wu [28], Wu and Shao [67]) and it is sat- 
isfied for many nonlinear time series. In particular, for the iterated random 
function (Elton [19] and Diaconis and Freedman [14]) r]n = -R(^n-i) En)) let 

\R{x,e)-R{x',e)\ 

Le = sup . — — 

be the Lipschitz coefficient. Then (33) is satisfied if lE(i^f) < 1 and E[|i?(xo, 
e)!"^] < oo for some xq. Wu and Min [66] showed that (33) holds for GARCH 
processes. 

Corollary 5. Let 2 < q<4 and assume (33). 

(i) Assume J2'^nWi\ = Oilog'^^'^ n). Then the LIL (16) holds. 

(ii) Assume Y.Zn\ai\= oln^''^-^''^ /\ogn). Then the SIP (18) holds. 

4. Proofs. 

Proof of Theorem 1. (i) Consider two cases 1 < g < 2 and q> 2 
separately. For 1 < g < 2, since VkSn, k = —oo,...,n — l,n, form martingale 



STRONG INVARIANCE PRINCIPLES 
differences, by Burkholder's inequality, 

I n n 
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\k=—oo 



k=—oo 



n 

< {K-k,q-^-k,qf 



k=—oo 



where we have applied the inequalities (|ai| + |a2|H )*''^ < |ai|'^/^ + |a2|'^^^ + 

• • • and ll'Pfc'S'nIlij < ^n-k,q — ^-k,q- Clearly Bq can be chosen to be 1 if g = 2. 
If (7 > 2, then || • \\g/2 is a norm and by Burkholder's inequality 



{\\Sn\\q/Bqf< ^ 



k=—oo 



n 

< Y mSn)\/2 
q/2 k=—oo 

n 

2 



n n 

= E \\'PkSnfq< E iK-k,q-A-k,q?. 
k=—OQ A-i^— oo 



fc=— oo 



(ii) By (2), Dk € L'^. Let Rk = Sk — M^. By the triangle inequality, for 

1 < j <ra, \\VjRn\\q < @n+l~j,q and, for j < 0, \\VjRn\\q < <dl^j^q-<dn+l-j,q- 

We first consider the case q>2. Since i?„ = X]j=-oo 'Pj^n and the summands 
f^vv^ ^^^^-;^n.^^^ differences and VjVk = if j ^ k, again by Burkholder's 



form martingale 
inequality, 



(34) 



7t 



11/2 



Lj=-oo J 

;ether with the triangle inequality || J2'j=-oo ^j\\q/2 ^ J2]j= 



which togi 
yields 



-oo 



l-^illg/2 



II n ||2 n 

^< E \\VM\l+T.mRnfq 
1 j=-oo j=l 

—n 

< E (©l-J,? - Qn+l~j,qf 



j=-co 



+ E (01-. 

j=-n+l 



,q ©n+l-j.q) 



' + E ®n+l-j,q 



—n n 

< E (ei-,„-e„+i_,,,)e„ + 2Ee^+i 

j=-oo j=l 
n n 

< nelg + 2 E 0n+i-,,9 < 3 E ©L- 
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The other case in which 1 < g < 2 is similar and it follows from (34) and 

n n n 



E 



]_] = -00 



J=-CO 





j=-oo 



j=-oo j=l 



^3E0L- 

Combining the two cases, we have (3). 

(iii) For j > let Z^-^ = ELi^-jf/fe)- Let p = q/{q - 1). We apply 
McLeish's [36] argument. Again we consider two cases 1 < q <2 and q> 2 
separately. Let q> 2. Notice that for a fixed j, Vk-jdiCk) form stationary 
martingale differences in k = 1, . . . ,n. By Burkholder's inequality and the 
triangle inequality, 



(35) 



k=l 



q/2 



< Bl E II [^k^A^k)]X/2 = Bln\\V,g{i,)\\l 

k=l 



If 1< g < 2, then 



(36) E 



Lk=l 



q/2 



<Y.n\rk^M^k)\Y' = n\\Vo9m\l. 



k=l 



By Doob's inequality (cf. Hall and Heyde [23], Theorem 2.2, page 15), 
\\maxk<n\Zj^k\\\g < p\\Zj,n\\q- By (35) and (36), \\Zj^n\\q < BgU^/''' \\Vog{Cj)\\q- 
Since Sk = J2j^o Zj,k and 



ii^:ii.<E 

j=0 

we have (4). □ 



max \Zi h\ 

k<n ' ^' ' 



< J2pB,n'/'^'\\VogmU=PBqn'^''eo,q, 

9 j=0 



Proof of Proposition 1. (i) Let p = q/{q- 1) and A = J2r=oK''^ 
where 



A, 



■2d-r 

E ll'^2''m — "S". 
.m=l 



2^{m-l) \\q 



-l/(p+<?) 
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For the positive integer k <2'^, write its dyadic expansion k = 2^'^ -\ h 2^-' , 

where < rj < ■ ■ ■ < ri < d, and k{i) = 2'^'^ -\ h By Holder's inequahty, 



\Sk\'< 



3 



Li=l 
j 



< 



.i=l 



q/p 



i=l 



i=l m=l 
r=0 m=l 

which entails II^^^H^ < A^^pJ^Lo KK^^'' = and hence (6). 

(ii) It easily follows from the argument in (i) since supg|S'fc^6i| < 

Y^i=iS^P9\Sk{i),e — Sk{i-i),e\- 

(iii) It suffices to prove (8) with Aq < oo. Let rj = (2-J ||52i Wq)^^'''^^^^ ^g, 
j > 0. Then X^j^o^j ~ ^- argument in (i), we have 



oo d 



^P(5*.>2'^/«J)<^5:i 

d=0 d=Ojr=0 



max 



\S2.m-S2J(m-l)\>'i''''Srj 



^ E E 2'^~^'IP(|52J I > 2'^/Hrj) 

j=Od=j 

OO OO 

^ E E 2'^F[2--''|52,|''((5rj)"'' > 2^=] 

j=OA:=0 

oo 

< ^2E[2-J|52.f (5r,-)-^] = 25-'?Af \ 

j=0 

where the inequality J]^o2'=P(|Z| > 2^=) < 2E{\Z\) is applied. □ 

Proof of Theorem 2. (i) Recall Dk = EZk ^fc9(-^i), Mk = Ei=i A 
and Rk = Sk- M^. Let p = q/{q - 1), a = 1/2 - 1/g > 0, A^- = 2°j/2 and 
i'k = 02*= ,g- Note that Qn,q is nonincreasing in n. By (3) of Theorem 1, 



/ 2^ \ 1/2 / j X 1/2 j 

\\R2AU<c[Y.Ql] <c[Y^ 2Vf <cY. 2^/^. 



Vm=l 



i=0 
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where C is a constant that does not depend on j and it may vary from Une 
to hne. So 



^ 2(^-J^)/'?||i?2. \\q < 2('=-J')/« 



j=Q 



3=0 

k 



J 

z 

i=0 



i=0 j=i 
k 

< C2'=/'?^2°^V'i- 



i=0 



By Holder's inequahty, 



1/9 / k 



1/p 



1/9 



4 = 



u=0 



u=0 



Hence by (15) and Proposition 1, 
1 



E 



max 

i<2'' 



- ^ ./ok 



^—akq / 



\i=0 



■3=0 

k \ 



00 00 r^-akq 



1=0 A;=max('t,3) 



(logyfc)9/2 



n~aiq 

<C7 + CEA?V'.'7f-w7 



< 00. 



By the Borel-Cantelh lemma, maXj<;2fe = Oa.s.lV^^ log A;), which com- 
pletes the proof by Stout's martingale LIL. 

(ii) By Corollary 1, i2„, = Oa.sXv^)- LIL easily follows. □ 

Proof of Theorem 3. (i) As in Theorem 1, let Mk = Z]i=i A and 
R^ = Sk- Mk. If 2 < g < 4, by (3) of Theorem 1, ||i?„|| = 0[n^/'?(logn)-i]. 
By Proposition 1, 

<^2('^-^-)/'^||i?2.||, 
q j=o 

1/2 



max I I 



< 



2-' 



l+^i2/.-l(logi)- 



0(2"'/nogd). 
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Hence we have max^<2d = Oa.s.[Xg(2'')] and, by the argument following 
(12), Rn = Oa.s.[X(}('^)]- The case in which g > 4 similarly follows. 

Now we shall deal with the martingale part and show that (18) holds 
with Sn replaced by M„. To this end, we apply Strassen's [57] martingale 
embedding method. Let 

n 

Jn = Y.[nDl\J^k-l)-CT% 
k=l 

If 2 < g < 4, let g{Tk) = E(Z)^^JJ^fc) - cj^, then g{Tk) G L'i/'^ and (17) implies 
Jn = Oa.s.(f^^'''^); see Remark 7. By the martingale version of the Skorokhod 
representation theorem (Strassen [57], Hall and Heyde [23]), on a possibly 
richer probability space, there exist a standard Brownian motion B and 
nonnegative random variables ri, r2, . . . with partial sums Tfc = X)i=i "^i such 
that, for /c > 1, = B(Tfc), E(rfc| J"fc„i) = E(D||J^fc-i) and 



(37) 



almost surely, where Cq is a constant which only depends on q. Let Qn = 
YJk=i[Tk - E(rfclJ^fc-i)]- Since E(t«/2) < oo and 1< q/2 < 2, by the martin- 
gale Marcinkiewicz-Zygmund SLLN, Qn = Oa.s.("'^^'^)- Notice that 

n 

r„ - na^ = Qn + ^[E(Tfc|.7^fc_i) - ¥.{Dl\J^k-i)] + Jn- 

k=l 



Therefore T„ — no"^ = Oa.s, 

,(^2/9), and 
max|B(Tfc) -B((t2A;)| < max 

k<n ' ^ 

(38) 



k<n 



sup I 

X : \x — a^k\<n'^i' 1 

i/«(logn)i/2i 



l{x)-M{a^k)\ 



n 



If > 4, by (ii) of Theorem 2, (17) implies the LIL 
J, 



lim sup ± 



k=0 



k=l 



< OO, 



where tn = \/2nloglogn. By the martingale LIL (Stout [54]), Qn = Oa.s.(in)- 
So there is a constant C < oo such that |r„ — nu^l < Ct,i almost surely and 
similarly 



max |B(rfc) - M{a^k)\ < max sup 



,1/21 



|B(x) 
0[X4(n)] 



holds almost surely. 
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(ii) As in (12), we have by Proposition 1 that the condition Qn,q* = 
0(„i/9*-i/2) implies i?„ = Oa.s.[v(^)]- By Corollary 2(ii), let i{n) = 
(logri)^/'?*(loglogn)(^+'^)/'?* (see also Remark 8), (19) entails Jn = Oa.s. [?^^^'^* x 
i{n)], which yields the desired result in view of the proof of (i). □ 

Proof of Theorem 4. Without loss of generality let < rj <1 — 2/q. 
By (3), ll-Rnllg = {n^~'^'^ + logn), which entails Rn = Oa.s.("-^^^) by the ar- 
gument in (12). We now deal with M„. By Corollary 3(i), the condition 
||E(D2|^o) - = 0{n-^) implies 

n 

Jn = J2[EiDl\J^k-i) - a^] = Oa.s.K(logn)i+2/'?(loglogn)4/'?]. 

k=l 

So the desired result follows from the argument of (38) in the proof of The- 
orem 3. □ 

Proof of Proposition 3. (i) Let i>k + l. Then we have the identity 

E[h{i,^i)\Ck]=nHii^i)\c^uej,i<j < k] 

(39) 

= E[h{Ci-iM-i,ej, l<j<k]= E[/i(e._i)|^fc]. 
Hence by Jensen's inequality, 

ak = \\n9iCk+iMk]-ng{ik+iMk]\\g 

< \\E[g{Ck+im-ngiik+i)\mg + \\n9iik+im-n9ii^^^^^ 

< MCk+i) - 9{ik+i)\\g + \\E[g{^k+i)m - E[g{ik+i)\ik]\\, < 2k+i- 

The inequality a^. < 2/3^_,_^ can be similarly proved. 

(ii) We only consider (27) since (26) can be similarly established. Ob- 
serve that E[g{Cn)m =nh{Cn-i)m and E[5(e„)|e_i] = E[/i(e„-i)|Co]. By 
Jensen's inequality, \\'Pog{(,n)\\q = ||E[/i(^„_i) - /i(|n-i)|Co]||g < "n-i- Since 

iE[/in(lo)ieo]=iE[/in(eo)ie-i], 

\\Vo9{Cn)\U = \\hn{Co)-nK{ioMo]\\g < IKiCo) " Kmir 
On the other hand, 

WKiCo) - hn{io)\\q < \\hn{Co) " ^n+l(?-l)llg + ll^n+l(C-l) " ^n(Co)||<7 
= 2\\Vo9{Cn)\U. 

Combining the preceding inequalities, we have (27). 



STRONG INVARIANCE PRINCIPLES 23 

(iii) Under the proposed conditions, Dk = Y^'^k 'Pkdi^i) £ L'^ by (i). Write 
Dk = D{e,k) and Vkg{ii) = Hi_k{ik), where D{-) and Hi_k{-) are measurable 
functions. We first show (29). Note that for A; > 1, E(L>^|^_i) = E[D(^fc)^|Co]- 
Since g/2 > 1, by Jensen's and Schwarz's inequalities, 

\mDi)\\^/, = mmk? - D{i,f m\\,/2 

<\\D{^^)-D{^k)\Umik) + D{ik)\U 
<2c,\\D{ik)-D{ik)\\r 
Notice that D{ik) = Y.T=kHi^k{ik). Then 

\\D{ik) - D{lk)\U < WHoiCk) - Ho{ik)\\g 

oo 

+ X! \\Hi-k{S,k) - Hi_k{^k)\\q. 
i=k+l 

Clearly, we have by (i) that 

WHoiCk) - Ho{ik)\\g < M^k) - g{ik)\u + wm-i) - h{ik-i% 

= Pk + ak-i < 3(3k- 

To show (29), it remains to verify that \\Hi^k{Ck) — Hi^k{£,k)\\q < 4dj_i holds 
for i > A: + 1. To this end, since Hi_k{^k) =nK^i.i)\Ck] - E[h{Ci_i)\^k-i] 
and //,„,,(efe) =E[/i(eVi)|l;^.] -E[/i(eVi)|ffc-i], 

m^kiCk) - H,^kiik)\u < mm^.iMk] - E[hii,.,Mk]\\, 

(41) . , 

+ iiE[Me.-i)ia^^i]-E[/i(e.-i)i4-i]ii,. 

Since i> k + 1, by (39) and Jensen's inequality, we have E[h{£,i-i)\£,k] = 
nhi^i.i)\ik] and 

l|IE[M?.-i)|4]-E[Ml.-i)|ffc]||, 

<\\E[hiC^-lMk]-nhii^-lMk]k 

(42) +\\nh{ii-iMk]-nh{ii^,)\ik]\\, 

< a,_i + \\E[h{Ci_i)\^k]-nh{ii-iMk]\\g 
<2ai_i. 

Similarly, ||E[/i(e.-i)|a-i] - E[/i(|i_i)|efe-i]|| < 3Qi_i. By (41), \\Hi_k{^k) - 
H^-k{L)\\q < 4a*-i and (29) follows. 
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The same argument also applies to (28). We need to replace ^j, and 
i5i by the * versions a* and (5*. Then \\Ho{ik) - HQ{Ck)\\q < and 
\\Hi-k{ik) — Hi-k{i1.)\\q — ^old for i> k + 1. Notice that we also have 

\\H,^k{^k) - Hi.k{Ck)\\g < \\Hi-kmU + \\H^-k{Ck)\U < 2ai-fc-i. 
So (28) follows from (40). □ 

Proof of Corollary 5. (i) By Theorem 2, it suffices to verify (15). 
Observe that (33) implies H'Po^nllg < Cr". Therefore, 

oo oo m oo oo 

m=n m=n i=0 i=0 m=max{j,n) 

n — 1 oo oo oo 

(43) =cE E i«— ii^' + '^EE 

i=0 m=n i=n m=i 

n—1 oo 



E r'Ollog'^/^n - i)] + E 0{r') = 0{\og' ^'-n) 



i=0 i=n 

which implies Qn,q — 0(log^^ n) and consequently (15). 

(ii) Under the condition on (a^), we have Qn,q* = 0[nV9*-i/2(iogn)-i] in 
view of the argument in (43). So the corollary follows from (i) of Theorem 
3 and Lemma 1. □ 

Lemma 1. For the process (32), assume Yl^QWi\ < ^ o.'^d (33), 2 < 
g<4. Then (17) holds. 

Proof. For n>k, let Vki]n = Ln-k{£,k)- Similarly as in (42), we have 

\\Ln~k{Ck) - Ln~.k{(,k)\\q 

< - Hr,*n\Ck)\\q + \mVn\Ck-l) " nV*n\Ck-l)\\q 

<M\Vn-V:\\q = Oir-/'^) 

in view of (33). Let p = r^-?. Since Vkg{Cn) = ET=o aj'Pk^n-j = E"=o «i x 
Ln-j-k{^k) and Dk = D{^k) = En=k'Pk9{^n), 



oo 



Eii^(ao-^(eDii.<EE 

k=l k=ln=k 

oo oo n—k 



n—k 

E "■ji^n-j-kiS.k) - Ln-j-k{(,k)] 
j=0 



<EEEKi^"-^' 

fc=l n=k j=0 

oo oo 

<Ep'(i-p)~'EI%-I <°°' 

k=l j=0 
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which entails (17) since \\E[D\CkMo] - cT^\\q/2 < 2\\Dk\\g\\D{Ck) - DiQWg- 
□ 
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